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ABSTRACT 



A study was conducted to determine the best method for 
obtaining information on teachers' backgrounds for the Schools and Staffing 
Survey (SASS) . Two alternative methodologies were compared: the collection of 
teachers' self-reports of their academic qualifications and the use of their 
college transcripts. While transcripts are assumed to provide more accurate 
information, their collection and analysis is more complex. The 1990-91 SASS 
relied on 637 completed interviews with teachers, 592 of whom had transcript 
information available. Several different kinds of teacher error were detected 
when the transcript information and self-reports were compared. Item 
nonresponse was sometimes significant, and some errors of omission were 
found. Some errors showed bias on the part of the respondent. Some errors 
apparently simply reflected differences in judgment. However, it was noted 
that transcripts themselves presented problems, particularly in the amount of 
work required for processing. It is concluded that self-reports and 
transcripts each have advantages and disadvantages, and that the 
determination of the kind of data to collect will depend on the quality and 
type of data to be collected, the resources available, and such 
considerations as the amount of burden to be placed on the respondent. 
(Contains 12 tables.) (SLD) 
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Introduction 



Many elements of our education system are built on the assumption that the characteristics 
of teachers are important in determining educational effectiveness. Thus, the higher education system 
includes extensive training in teacher education, and a system of teacher certification is designed to 
further assure that teachers will have the necessary qualifications. While desirable in concept, 
however, measuring teacher quality has proved difficult in practice. Teaching styles vary widely, and 
the wide variation in student needs can make it difficult to specify an ideal set of qualifications even 
within a single classroom. 

One widely assumed component of teacher effectiveness is the extent of teacher 
preparation in both teacher education and in the specific subject matter being taught. The type of 
degree earned, the teacher's major and minor, the number and type of courses taken in specific subject 
areas, and the grades earned all might be considered potential indicators of a teacher's training, and 
might be used either to guide teacher selection or to modify teacher education requirements. 

This study is a methodological study designed to determine the best method for obtaining 
information on teachers' backgrounds. Specifically, two alternative research methodologies will be 
compared: the collection of teachers' self-reports of their academic qualifications, as provided on 

survey questionnaires, and the use of teachers' college transcripts. Transcripts are assumed to 
ultimately provide the most accurate and complete descriptions of teachers' backgrounds, since they are 
not subject to potential reporting bias of teachers who may wish to inflate their qualifications, and they 
provide a degree of detail that might not be possible when dependent on teachers' recall: however, the 
collection and analysis of transcripts is administratively more complex than the administration of a 
survey questionnaire. 

The 1990-91 Schools and Staffing Survey (SASS) was designed to collect teachers' self- 
reports of their degrees earned, their majors and minors, the number of courses or credits taken in 
teacher education and in the teacher's two main teaching areas, and the number of courses taken in 
science and mathematics (among those teachers who taught at least one course in science or 
mathematics). 1 Teachers were also asked to list all of the colleges they had attended, so that 



'The sample was split in half between teachers who were asked to state the number of courses taken in teacher education and their two main 
teacher areas, and teachers who were asked to state the number of credits. 



transcripts could be collected to verify the accuracy of the data provided by the teachers. The U.S. 
Bureau of the Census conducted the initial mail and telephone survey of the teachers, and Westat, Inc. 
conducted the transcript portion of the study. 



Data Collection 

In order to select the SASS sample for 1990-91, 200 schools were chosen in 10 states; 
there were 50 public elementary schools, 50 public secondary schools, 50 private elementary schools, 
and 50 private secondary schools. Of the 200 schools, 174 schools were ultimately determined to be 
eligible for the study and agreed to participate. A sample of 867 teachers was next selected at the 
schools, with no more than 5 teachers from any one school. A total of 32 teachers were later 
determined to be out-of-scope, leaving a total of 835 eligible teachers (Table 1). Based on 637 
completed interviews, the final response rate for the teacher questionnaire was 76 percent. 

Of the 637 responding teachers, 45 either refused participation in the transcript portion of 
the study, or failed to supply any information on which colleges they attended. According to the 
Federal regulations (45 CFR 99.31) that implement the Family Education Rights and Privacy Act 
(FERPA) (20 U.S.C. 1232g), transcripts may be provided to the U.S. Department of Education or its 
authorized representatives even without the prior consent of the students. 2 For this study, 
nevertheless, any teachers who specifically refused participation were left out of the transcript study. 3 
This left a total of 592 teachers. 

Teachers were asked to list all colleges (both undergraduate and graduate) that they 
attended, whether or not they graduated from those colleges, and transcripts were sought from all 
colleges listed. Based on the information provided by all teachers, including those who refused 



2 However, the handling and analysis of the transcripts must be performed in a way that protects the confidentiality of the students. 

3 It is possible that some teachers might have refused participation in the transcript study because they were not proud of their academic 
qualifications, and they were afraid that the transcripts would show that their self-reports overstated their credentials. In such a case, there 
is a possible bias from the exclusion of refusals. Data from the SASS teacher questionnaires were used to partially test this hypothesis, but 
the findings were mixed. There is weak evidence that teachers who refused participation had weaker backgrounds than participating 
teachers: 30 percent of teachers not in the transcript study said they had master's degrees, compared with 37 percent of those in the study, 
and 20 percent reported one undergraduate course or leas in teacher education, compared with 14 percent of those in the study. (However, 
neither relationship was statistically significant.) Thus, one could argue that teachers with lower academic qualifications are more reluctant 
to have their academic records reviewed. However, this does not necessarily mean that the teachers attempted to overstate their 
credentials. The fact that their self-reports could be used to show a difference in academic qualifications might be interpreted as indicating 
they were willing to report their backgrounds accurately, without inflating them. 
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participation, 1,985 separate transcript requests were identified; of these, 130 were excluded because 
the teachers refused participation in the transcript study, 14 because the attendance was at a foreign 
institution, and 6 because the school could not be located based on the information supplied by the 
teacher, leaving a total 1,835 requests. A total of 1,524 transcripts were received, including 1,356 
transcripts among those that were originally requested, and an additional 168 transcripts reflecting an 
undergraduate or graduate enrollment that had not been indicated on the SASS questionnaires. 4 
Additionally, schools sometimes responded in ways other than sending a transcript: in 53 cases, a 
school indicated that a teacher had never attended, and in 81 cases the school responded that it was 
unable to locate the teacher's transcript (e.g., because they had insufficient information about the 
teacher, or because of difficulties in retrieving records that were relatively old). The total number of 
school responses was 1,658 out of 2,003 identifiable transcript requests (1,835 original requests, plus 
169 transcripts received that had not been anticipated), or 83 percent. 5 

Another way of examining the response rate is to determine the amount of data received 
for each teacher. At least one transcript was received for 92 percent of the teachers. However, while 
a single transcript was sufficient to provide complete data for some teachers, other teachers had up to 9 
separate transcript requests. For those teachers with multiple transcripts, complete data could 
potentially be obtained in several ways: most typically, all transcripts requested were received, but in 
some cases the availability of transfer information on one transcript might allow a complete picture of 
the teacher's academic record, even if the transcript were never received from the institution where the 
courses were originally taken. Also, in those cases where schools indicated that a teacher had never 
attended, assuming that the school was correct, it was potentially possible to develop a complete record 



*Th is latter situation occurred when teachers attended the same college both as an undergraduate and as a graduate, but did not indicate both 
statuses on the SASS questionnaire, in such cases, only a graduate or undergraduate transcript would have been requested (depending on 
which was indicated by the teacher), but not both. (The distinction between undergraduate and graduate status was made because 
institutions often treat graduate transcripts as separate transcripts, sometimes even processing those transcripts in a separate office.) It is 
likely that in some cases, among those institutions that handle graduate and undergraduate transcripts in separate offices, the dual 
attendance status of the teacher was never identified. However, other institutions did send both graduate and undergraduate transcripts, so 
that sometimes more transcripts were received than were requested. 

^Another 94 transcripts were received after the close of data collection, and were not coded. If these transcripts had arrived earlier, the total 
response rate would have been 87 percent. The calculation of a response rate is complicated because of problems in defining both the 
denominator (the total number of requests) and the numerator (the number of responses received). In defining the denominator, the rule 
that was followed was to treat all listings of a college by a teacher as a single transcript request unless the teacher took both undergraduate 
and graduate courses from the same institution, in which case the period of graduate enrollment was treated as a separate transcript request 
from the undergraduate enrollment. If a new transcript was identified (e.g., because an institution mailed a graduate transcript, even 
though the teacher had only indicated undergraduate attendance at that institution), that transcript was also treated as a transcript request to 
be included in the denominator. (Otherwise, it would be possible to have response rates above 100 percent.) In defining the numerator, 
the complication is that some institutional responses might be considered as legitimate responses even though no transcript was received: 
for example, if an institution reported that a teacher had never attended, the institution provided all data that were available, and the data 
were sufficient to make a judgment about the quality of the teacher-supplied data, even without the receipt of a transcript. 
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of a teacher's college attendance even without a transcript from that institution. 6 Allowing for these 
various possibilities, complete data were obtained for 51 percent of the teachers. 7 

The importance of collecting only partial transcript data for some teachers depends on 
which transcripts were missing, and on the type of information being verified. For example, often a 
receipt of a degree can be verified even if some transcripts were not received; also, the receipt of a 
baccalaureate degree might legitimately be assumed if other transcripts indicate the receipt of a more 
advanced degree. Even when counting the number of courses or credits taken in a subject area, partial 
transcript data are sometimes sufficient to establish a lower bound that is adequate for verifying the 
teacher's response on the SASS questionnaire. (For example, a teacher's claim to have taken at least 
four undergraduate courses in teaching methods might be verified even if only a lower bound is 
known; similarly, a claim to have taken less than four courses might be rejected if the lower bound is 
four or higher.) Thus, statistics in this report will often include references to partial data where those 
data are sufficient for the analysis. 

Most teachers (349, or 59 percent) gave their signed permission to collect the transcripts, 
while the remaining teachers supplied information that could be used to collect transcripts, but did not 
provide a signed permission form that could be sent to their colleges (Table 2). 8 If teachers did not 
supply signed permission to collect their transcripts, colleges were sent a letter from the National 
Center for Education Statistics and a copy of the Federal regulations to indicate that signed permission 
forms were not required to satisfy Federal privacy requirements. Most colleges were willing to 
provide transcripts based on this information, but some were not; the net effect was a somewhat lower 
response rate if no signed permission were available. Among those teachers who gave their signed 
permission, all transcripts were obtained for 51 percent, while among those for whom no teacher 
signatures were available, all transcripts were obtained for only 37 percent. 9 Also, for some teachers 



6 A problem is that since some institutions had difficulty in locating transcripts, the report that a teacher never attended a college sometimes 
reflected the college's inability to locate the record rather than the teacher's nonattendance. Schools were asked to differentiate between 
records they could not locate and teachers who never attended, but their responses were not always accurate: sometimes a teacher's 

attendance could be verified through transfer data at another institution even when the first institution said the teacher had never attended. 

^ In order to provide a complete description of the sample. Table 1 also presents the response rates for public versus private schools, and for 
the two questionnaire types (courses versus credits); however, there is no theoretical reason to expect different response rate based on these 
characteristics, and in fact response rates varied only slightly. 

^In some cases, teachers were administered the questionnaire over the telephone and gave verbal permission to collect transcripts, but no 
signatures could be provided. 

^The relationship between the receipt of transcripts and the signed permission of the teachers was statistically significant at the .05 level. 



the lack of signed permission was associated with an inability to collect transcripts from any of the 
institutions they had attended: this failure to collect any transcripts appeared for 14 percent of the 
teachers who failed to provide signatures, but for only 4 percent of those teachers who did provide 
signatures. 



Figure 1 shows the percentage of teachers that gave their signed permission to collect 
transcripts, based on school characteristics and the type of data provided. The two school 
characteristics that were examined did not show differences that were statistically significant, with 
teachers at public schools and elementary schools only marginally more likely to give signed 
permission (52 percent versus 48 percent for public/private schools, and 60 percent versus 50 percent 
for elementary /secondary schools). However, both measures that were related to the type of data that 
teachers provided did show statistically significant results. Teachers who provided data on the number 
of courses taken were more likely to give permission than teachers who provided data on the number 
of credits (60 percent versus 47 percent), and teachers who reported having a master's degree were 
more likely to give permission than teachers who said they had no master's degree (60 percent versus 
51 percent). The reasons for these differences cannot be established from the survey questionnaire, but 
it is possible that teachers who are proud of their academic records (e.g., have advanced degrees) are 
more likely to give permission, and that teachers who are confident of their responses (i.e., if it is 
easier to state the number of courses than the number of credits) are more likely to give permission. 



Teachers' Self-reports of the Schools they Attended 

Teachers were asked to list every college or university they had attended, whether or not 
they obtained a degree at the college, with space provided for nine colleges. This question was asked 
in order to facilitate the collection of college transcripts rather than to verify teachers' accuracy in 
reporting; nevertheless, the teachers' responses can be examined for accuracy. Two different types of 
errors are possible: teachers may list colleges they never attended, or fail to list colleges that they had 
attended. 



Overall, there were 53 cases (from 44 teachers, or 8 percent of the total number of 
teachers for whom at least one transcript was collected) in which a teacher contradicted a college, with 
the teacher reporting attendance but the college stating the teacher never attended. A total of 7 cases 
could be identified as errors on the part of the college (through transfer notations on other colleges' 



transcripts), while the remaining 46 cases (from 38 teachers, or 7 percent) may indicate false reports 
by the teacher. Additionally, there were 81 cases (from 67 teachers, or 12 percent) in which colleges 
were unable to locate the teacher's transcripts; for 6 of the cases, attendance at the school could be 
confirmed through transfer notations on other colleges' transcripts, but this left 75 cases (from 61 
teachers, or 11 percent) that might represent false reports. Aside from the data limitations noted 
earlier, some of these "false reports" may be due to differences in definitions of college attendance. 
For example, in one case a personal communication from the teacher indicated that the teacher had 
taken noncredit courses at the college; the teacher listed the college on the SASS Questionnaire because 
the teacher felt the courses enhanced her perspective as an educator, while the college does not 
maintain transcripts for such noncredit courses. 

The other type of error - a failure of the teacher to list all colleges attended - is more 
difficult to identify. If a teacher listed no colleges at all, then no colleges were contacted for 
transcripts, and no data are available to confirm or deny the teachers' academic background. If a 
teacher gave only a partial list of colleges attended, on the other hand, there is some chance of 
detecting the error: other colleges' transcripts may include transfer credits from the missing college(s). 
For 49 teachers (9 percent), transfer information on the transcripts indicated that a total of 53 
additional colleges had been attended besides those listed on the questionnaire. These errors might be 
attributed either to poor memory on a teacher's part, or in some cases, to the relative unimportance of 
the teacher's attendance at the college (e.g., a teacher may have taken only a single course during the 
summer, and failed to list the college for that reason). 

In short, while the list of colleges attended was not meant as an item for verifying 
teachers' accuracy, a relatively substantial number of errors were detected. Further, only confirmed 
errors are reported here, so these estimates provide lower bounds on the number of errors. It may be 
that if additional colleges were contacted, additional omissions might be detected. One practical 
implication is that when transcript studies are conducted, the lists provided by teachers or other 
respondents should not be considered as fully accurate. One possibility is to change the wording of the 
questionnaire to reinforce that all colleges should be listed (e.g., the questionnaire might specifically 
mention to include colleges where summer courses were taken), while another is to seek additional 
sources of information on what colleges were attended (e.g., by inspecting transcripts for notations on 
additional colleges and then contacting those colleges for transcripts, or by examining administrative 
records at the schools where the teachers are located). 
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Teacher Item Response Levels 



As noted in the previous section, errors can occur either through teachers providing 
incorrect responses, or through their failure to provide (complete) responses. At times, these two 
types of errors might be independent (e.g., a teacher might accidentally skip an item, but be willing 
and capable of providing a correct answer), while at other times they might be interrelated (e.g., a 
teacher may skip an item because of an inability to provide the information, or a teacher may respond 
to an interviewer's persistence for a response by manufacturing a reply or by making a rough 
estimate). There is no necessary rule as to which type of error is worse: this may depend on the 
magnitude and degree of bias of the incorrect responses. This section will discuss teacher "errors" 
associated with non-response, while the remaining sections will examine the accuracy of teacher 
responses on the SASS questionnaire by comparing them with data from the postsecondary transcripts. 

Technically, non-response may occur either through questionnaire non-response, where a 
teacher fails to respond to an entire survey questionnaire, or non-response to individual items on the 
questionnaire. However, because the overall response rate to a survey depends so significantly on the 
methods and extent of followup, it is difficult to compare the overall response rates for the SASS 
questionnaire and transcript data collection. 10 For this reason, this section will focus on item non- 
response, and especially on comparisons of non-response rates among different items to distinguish 
which types of data are most difficult to collect. Even with this limitation, caution should be applied 
because of the great variability of non-response from one survey to another. For both overall non- 
response and item non-response, the level of non-response will depend on the actual design of the 
questionnaire, the mode of administering the questionnaire (e.g., by mail or by telephone), the degree 
and methods of followup for non-response, the level of burden presented by the questionnaire, the 
resources available for survey administration, and other aspects such as the degree to which the 
questionnaire includes questions on sensitive topics, and the degree to which the potential respondent 
perceives the survey to be useful. 

Table 3 shows that item response rates varied depending on the level of detail requested. 
The item response rates were highest for general information about degrees earned (i.e., 97 percent or 



10 However, it should be noted that 24 percent of the eligible teachers failed to respond to the survey, and that obtaining high response rates 
appears to getting more difficult for all surveys. There may be situations where, even without the help of teachers' signed permission to 
collect transcripts, and depending on the potential burden and intrusiveness of the questionnaire, the collection of transcripts may provide 
the best opportunity for high survey response rates. 



